Field of the Invention
This invention relates generally to a system and method for grid-based image resolution enhancement for video processing and, more particularly, to a system and method for providing a high resolution display image by de-warping a warped grid in each of at least two different camera views.
Discussion of the Related Art
Modern vehicles generally include one or more cameras that provide back-up assistance, take images of the vehicle driver to determine driver drowsiness or attentiveness, provide images of the road as the vehicle is traveling for collision avoidance purposes, provide structure recognition, such as roadway signs, etc. Current front view and rear view cameras on vehicles are typically wide field-of-view (FOV) cameras, for example, a 135° FOV. Wide FOV cameras typically provide curved images that cause image distortion around the edges of the image. Various approaches are known in the art to provide distortion correction for the images of these types of cameras, including using a model based on a pinhole camera and models that correct for radial distortion by defining radial parameters.
It has been proposed in the art to provide a surround view camera system on a vehicle that includes a front camera, a rear camera and left and right side cameras, where the camera system generates a top down view of the vehicle and surrounding areas using the images from the cameras, and where the images overlap each other at the corners of the vehicle. The top down view can be displayed for the vehicle driver to see what is around the vehicle for ease of backing up, parking, etc. Further, future vehicles may not employ rearview mirrors, but may instead include digital images provided by the surround view cameras.
In order to provide a surround view completely around the vehicle with a minimal number of cameras, available wide FOV cameras having a 135° FOV will not provide the level of coverage desired, and thus, the cameras will need to be ultra-wide FOV cameras having a 180° or greater FOV. These types of ultra-wide FOV cameras are sometimes referred to as fish-eye cameras because their image is significantly curved or distorted. In order to be effective for vehicle back-up and surround view applications, the distortions in the images need to be corrected.
Optimization of the display resolution of one camera is known, as described in Patent Application Publication No. 2014/0085409 A1 entitled, “Wide FOV Camera Image Calibration and De-Warping,” filed Mar. 15, 2013, assigned to the assignee of the present application and incorporated herein by reference. However, there is a need in the art to optimize the display resolution of multiple camera views that are created from multiple vehicle images and cameras.